The merits of using citation‐based journal weighting schemes to measure research performance in economics: The case of New Zealand

In this study we test various citation‐based journal weighting schemes, especially those based on the Liebowitz and Palmer methodology, as to their suitability for use in a nationwide research funding model. Using data generated by New Zealand’s academic economists, we compare the performance of departments, and individuals, under each of our selected schemes; and we then proceed to contrast these results with those generated by direct citation counts. Our findings suggest that if all citations are deemed to be of equal value, then schemes based on the Liebowitz and Palmer methodology yield problematic outcomes. We also demonstrate that even between weighting schemes based on a common methodology, major differences are found to exist in departmental and individual outcomes

The New Zealand performance based research fund and its impact on publication activity in economics

New Zealand’s academic research assessment scheme, the Performance Based Research Fund (PBRF), was launched in 2002 with the stated objective of increasing research quality in the nation’s universities. Evaluation rounds were conducted in 2003, 2006 and 2012. In this paper, we employ 22 different journal weighting schemes to generate output estimates of refereed journal paper and page production over three six year periods (1994-1999; 2000-2005 and 2006-2011). These time periods reflect a pre-PBRF environment, a mixed assessment period, and a pure PBRF research environment, respectively. Our findings indicate that, on average, research productivity, defined in either paper or page terms, has increased since the introduction of the PBRF. However, this outcome is due to a major increase in the quantity of papers and pages produced per capita that has more than off-set a decline in the quality of published outputs since the introduction of the PBRF. In other words, our findings suggest that the PBRF has failed to achieve its stated goal of increasing average research quality, but it has resulted in substantial gains in productivity achieved via large increases in the quantity of refereed journal articles

Research output in New Zealand Economics Departments 2000-2006

This paper considers the research productivity of New Zealand based economics departments over the period 2000 to 2006. It examines journal based research output across departments and individuals using six output measures. We show that Otago and Canterbury performed consistently well over the period, with Otago generally the highest ranked department. The measures used place different emphasis on ‘quality’ versus ‘quantity’. Which measure is used has a significant influence on the rankings of Auckland, Victoria and Waikato. The controversy surrounding the inclusion of ‘visitors’ and the influence of research stars is considered. Rankings of the leading individual researchers are provided

The excellence in research for Australia Scheme: An evaluation of the draft journal weights for economics

In February 2008, the Australian government announced its intention to develop a new quality and evaluation system for research conducted at the nation’s universities. Although the Excellence in Research for Australia (ERA) scheme will utilize several measures to evaluate institutional performance, we have chosen to focus on one element only: the assessment of refereed journal article output based on ERA’s own journal weighting scheme. The ERA weighting scheme will undoubtedly shape the reward structure facing university administrators and individual academics. Our objective is to explore the nature of the ERA weighting scheme for economics, and to demonstrate how it impacts on departmental and individual researcher rankings relative to rankings generated by alternative schemes employed in the economics literature. In order to do so, we utilize data from New Zealand’s economics departments and the draft set of journal weights (DERA) released in August 2008 by ERA officials. Given the similarities between Australia and New Zealand, our findings should have relevance to the Australian scene. As a result, we hope to provide the reader with a better understanding of the type of research activity that influences DERA rankings at both the departmental and individual level

The merits of using citations to measure research output in economics departments: The New Zealand case

In this paper we explore the merits of utilizing citation counts to measure research output in economics in the context of a nation-wide research evaluation scheme. We selected one such system for study: the New Zealand government’s Programme-Based Research Fund (PBRF). Citations were collected for all refereed papers produced by New Zealand’s academic economists over the period 2000 to 2008 using the databases of the ISI/Web of Science and, to a limited extent, Google Scholar. These data allowed us to estimate the time lags in economics between publication of an article and the flow of citations; to demonstrate the impact of alternative definitions of ‘economics-relevant’ journals on citation counts; and to assess the impact of direct citation measures and alternative schemes on departmental and individual performance. Our findings suggest that the time-lags between publication and citing are such that it would be difficult to rely on citations counts to produce a meaningful measure of output in a PBRF-like research evaluation framework, especially one based explicitly on individual assessment

Ranking economics departments in terms of residual productivity: New Zealand economics departments, 2000‐2006

This paper considers a new approach for ranking the research productivity of academic departments. Our approach provides rankings in terms of residual research output after controlling for the key characteristics of each department’s academic staff. More specifically, we estimate residual research output rankings for all of New Zealand’s economics departments based on their publication performance over the 2000 to 2006 period. We do so after taking into account the following characteristics of each department’s academic staff: gender, experience, seniority, academic credentials, and academic rank. The paper concludes with a comparison of rankings generated by the residual research approach with those generated by traditional approaches to research rankings

Results of the 1995 Illinois Groundhog (Woodchuck) Hunter Survey

ID: 875; Administrative Report, PR Project W-112-RReport issued on: June 27, 199

Bargaining and Trust: The Effects of 36hr Total Sleep Deprivation on Socially Interactive Decisions

Though it is well known that sleep loss results in poor judgment and decisions, little is known about the influence of social context in these processes. Sixteen healthy young adults underwent three games involving bargaining (‘Ultimatum’ and ‘Dictator’) and trust, following total sleep deprivation (TSD) and during rested wakefulness (RW), in a repeated measures, counterbalanced design. To control for repeatability, a second group (n=16) was tested twice under RW conditions. Paired anonymously with another individual, participants made their simple social interaction decisions facing real monetary incentives. For bargaining, following TSD participants were more likely to reject unequal-split offers made by their partner, despite the rejection resulting in a zero monetary payoff for both participants. For the trust game, participants were less likely to place full trust in their anonymous partner, again affecting final payoff. Overall, we provide novel evidence that following TSD, the conflict between personal financial gain and payoff equality is focused on unfavourable inequality. This results in the rejection of unfair offers, at personal monetary cost, and the lack of full trust, which would expose one to being exploited in the interaction. As such, we suggest that within a social domain a rational decision may not prevail over more emotional options following TSD, which has fundamental consequence for real-world decision making involving social exchange. Key Words: Sleep loss, trust, bargaining, social preference, interaction